NERL: Neural-Network Emulation of Reinforcement Learners

Neural networks are one of the most useful machine learning techniques, but typically require substantial data sets of input/output pairs. In this paper we define NERL, a method of training NNs on Reinforcement learner agents. We trained an approximate RL agent on the Space Invaders game in the Arcade Learning Environment (ALE) and on simulated robots in Pyrobot. We then trained NNs on the input given to the RL agent and the action chosen by the RL policy. In ALE, we compared three different methods of feature generation, ranging from a raw visual field to hard coded game-specific features. We found that in ALE, NNs were very good at mimicking the RL agent, and in one instance were able to outperform the RL agent in terms of score. The light foraging task proved too chaotic for NERL as the dual training compounded the noise in the environment. NERL shows promise as a technique to easily generate multiple machine learning agents without requiring a large amount of labeled training data.